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ABSTRACT 


Software testing are executed throughout all phases of 
software development life cycle (SDLC). It is though time 
consuming, labour intensive and now becoming expensive 
due to the used of an automated tool system. Errors and defect 
detection in software are identify using software techniques 
and tools. These techniques and tools are numerous and to 
improve on the software quality, there is the need to conduct a 
testing activity using a suitable technique so as to ascertained 
a product with an excellent quality features such as being; 
reliable, secure, efficient etc. In most cases, multiple 
automated techniques and tools are required to cover all the 
possible testing state, with the requirement of high skilled 
expertise, making it professional. The aim of this paper is to 
investigate on some kind of dynamic testing techniques 
(white, black and grey box) and an automated tools (load, web 
and mobile) given their comparative and evaluation analysis 
based on an established criterion. The conclusion of this 
investigation, has shown that choosing an appropriate 
techniques/tools increases test depth, software reliability and 
as well reduces their execution period with greater flexibility. 
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1. INTRODUCTION 


Software testing techniques is an approach adopted in testing 
for a particular software product throughout a software 
development life cycle (SDLC) [1]. According to [1],[2]-[4], 
software testing aids in defects/bugs detection during product 
execution and as well acts as an integral part of quality 
assurance. Most times, the products are validated against 
client’s requirements by software developers in delivering a 
defect free product. In order to test for the software product 
appropriately, it is vital to determine the types of tests 
required and when it is needed to be performed. 


There are many ways of testing for a software product and 
incorrectly chosen a testing approach may result in a failure to 
detect number of errors or late errors detection. Hence the 
needs to conduct a testing activity using a suitable technique 
so as to ascertained a product with the quality features such as 
being; reliable, secure, capable, efficient, portable, 
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functionable, reusable, manageable, maintainable etc. [5]. The 
ultimate aim of software development process is to produce 
high quality software which is defect-free. Bigger quality 
software has characteristics like high cost implication, 
consistent, user satisfactory and time efficiency [7]. 


The basic software testing techniques are Dynamic (white box, 
black box and grey box) and Static (Inspection, walkthrough, 
technical and informal reviews) with each having its unique 
attribute and functionality [5, 22]. Dynamic testing is an 
automated testing technique which is executed by a 
professional for proper analysis of the software dynamic 
relationship behavior. While Static techniques require no code 
and it is performed prior to code deployment for accurate code 
evaluation by a non-professional. Similarly, the testing tools 
are broadly categorized into Manual and Automated tools. 
While Manual testing tools are executed manually by the 
developers with the use of a test plan, an automated testing 
tools is said to overcome the manual challenges. Hence, the 
exponential rise in the number of companies embracing and 
developing automated testing tools for series of applications. 
Automated software is further sub-divided into Proprietary 
Software Test Automation Tools (PSTAT) which is licensed 
and otherwise known as commercial tool and Open Source 
Software Test Automation Tools (OSSTAT)which is free. 
This research work is aimed at investigating onto some 
popular software techniques (black, white and grey box) and 
automated tools (test complete, Selenium, JMeter and Hpload 
runner) and as well to carried out a comparative and 
evaluation analysis of both. 


2. SOFTWARE TESTING TECHNIQUES 


Testing techniques are used throughout the software 
development life cycle (SDLC) for developing a test case, 
detecting a fault, debugging an error and maintaining its 
reliability. There exist a numerous number of techniques, with 
each having its unique functions for sufficient test coverage. 
Combining two or more of these techniques is based on 
functional requirement that leads to the development of a 
qualitative and an error free software. Basically, there are 
three types of testing techniques (Dynamic); Black box, white 
box and the grey box [16]. 
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Figure 1. Software Testing Techniques 


2.1 White Box Testing 


The testing techniques is based on the knowledge of the inner 
logic of an application code and otherwise known as Open 
Box or Logic Driven box. The Internal functioning of a code 
and software is known and become accessible by the user. In 
this, test cases are being made based on the code. The 
applicable strategies to white box testing as defined by [9] are; 
units, integration and system testing. 


Whitebox 


(Application 
code) 


Figure 2. White-box testing 


2.2 Black Box Testing 


This is a type of techniques handling the tests based on 
requirements and functionality without the consideration of 
the internal system design and otherwise referred to as 
functional/non-functional testing. In this type of testing, the 
inner configuration parts are not known and then cannot be 
accessible by the user [16]. The information of the code is 
only required for the test cases design from the Input / Output 
without the test personnel knowing how the execution is done. 
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Fig. 3. Blackbox testing 


2.3 Grey Box Testing 

It is a testing technique which combine Black Box Testing 
and White Box Testing technique. It required a finite amount 
of knowledge on testing the internal working of an 
application. The main goal of using this testing approaches is 
to increase effective coverage of the application code. In this 
kind of testing, the test personnel have a partial understanding 
of the internal structure of the system under test. Hence the 
need for debugging software applications by making an input 
through the front-end, and verifying the data on the back-end 
[16]. 


Blackbox 


Whitebox 


Figure 4. Grey-box testing 


Table 1. Cons and Pros of Testing Techniques (white, black and grey box) 


Advantages (Cons) 


S/N White Box 


Black Box 


Grey Box 


1 Code knowledge is required 


Also offers code optimization 


Code knowledge is unknown 


Offers both white-box and 
black box testing benefits. 


2 Coverage of more test cases due to 
tester’s knowledge of the code 


Speedy test case improvement without 
any access to code 


Logically deal with complex 
test case scenario 


3 Reveals hidden code errors. 


Tester and Programmer are autonomous 
to each other. 


For independent testers and 
developers, boundary 
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remain constant. 


debugging tools and code analysers. 


at all. 


4 Approximates the partitioning done Testing is being performed from the view | In dim box testing, analyser 
by execution equivalence. point of users. can plan incredible test 
i situations 
Also, Test cases can be designed upon 
completion of specifications. 
Disadvantages (Props) 
S/N White Box Black Box Grey Box 
1 Expensive to maintain Challenging and limited test cases Complete white-box testing 
coverage without clear functional cannot be done due to 
specifications. inaccessible source code 
2 Specialized tools are required such as Some parts of the backend are not tested A defect association is 


difficult in distributed 
systems. 


3 Difficult to find and test all the code 
in a runtime environment. 


Difficulty to identify all possible inputs in 
limited testing time. 


Test scope restriction due to 
inaccessible source code 


4 Miss cases of omitted code 


Inefficient testing due to the limited 
knowledge of code. 


Numerous system ways 
stay untested. 


2.4 Levels of Testing 
[3, 4, 15, 22], proposed the following levels of testing 


a) Unit testing: It is a kind of white box testing otherwise 
referred to as components/module testing. It is the smallest 
testable part of an entire application that validate the 
functionality of a definite segment of code and as well satisfy 
the business requirements. 


b) Integration testing: It is a top-bottom approach with the 
code divided into an individual segment and tested as a group. 
The main task is to analyze parameters such as functional, 
performance and reliability requirements which are placed on 
major design items. 


c) Functional testing: It can be referred as black-box testing. 
The testing is done by providing validate input and thus 
outcomes are observed accordingly. Functional testing will 
test the entire functionality of the system by comparing 
expected output with actual output. 


d) Structural testing (white box): It mainly focus on the 
internal design of the program like path coverage, statement 
coverage, control structure and program complexity etc. 
thereby improving the design and usability. 


e) System testing: It is considered as more limited type of 
testing, that seeks to detect any defects within the software 
units that are integrated together. It is also a black box testing 
which is used to evaluate the system compliance against a 
stipulated requirement. 


(£) Acceptance Testing: It is a kind of black box testing 
which is done by the user to authenticate the business 
requirement conformance and the readiness of a product, 
service or system as part of a quality assessment measures. 
[1]. The user has no knowledge of the internal working of the 
system, so it is done upon the completion of the projects prior 
to handling over by the developers to the clients or users. The 


objective is to give utmost assurance of running an error free 
system with maximal reliability and efficiency [22]. 
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Figure 5. Levels of Testing 
3. SOFTWARE TESTING TOOLS 


Software testing tools are series of tools used for error 
detection. It is always a good practice to introduce them as 
early as possible in the software development lifecycle 
(SDLC). The tools are used to automate certain tasks, improve 
testing efficiency and discover issues that might be difficult to 
find using manual review alone. They inspect behavior of a 
program on a finite set of test cases (a set of inputs, execution, 
preconditions, and expected outcomes developed for a 
particular objective, such as to exercise a particular program 
path or to verify compliance with a specific requirement, for 
which valued inputs always exist) [2],[3],[4],[11], [6]. The 
software testing tool should meet an organization short and 
long term goals [6]. 


In this research; the testing tools were chosen for their crucial 
roles played in various automated organizational sectors 
across the nation, with their quality assurance consideration. 


26 


Software Testing Tools 


Proprietary 
& 
Open Source 


TestComplete Selenium 


Figure 6. Software Testing Tools 


Hploadrunner 


3.1 Proprietary Software Test Automation 
Tools (PSTAT) 


Proprietary Software Test Automation Tools are Commercial 
software’s developed by companies which require licenses to 
be purchased. Some of the top emerging proprietary software 
include HP, IBM, Oracle, Microsoft etc. It provides access to 
the software code to those users who had purchased the 
license or procured an agreement to use it [18]. 


3.2 Open Source Test Automation Tools 
(OSSTAT) 


Open Source Software Test Automation tool is an emerging 
market with companies offering testing tools free of cost and 
the Source code of the tools are available for 
modification/customization. Support is also available for these 
tools from Open Source development community [18]. 
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3.3 Web, Mobile & Functional Testing 


Tools 
It is a type of testing which verifies that each function of the 
software application and operation are in conformance with 
the requirement specification. This testing involves checking 
of User Interface, APIs, Database, security, client/ server 
applications and functionality of the Application under Test. 
The coverage tools are;( Selenium and Test Complete). 


3.3.1 Selenium 

Selenium is an open-source software automation tool 
developed by Apache 2.0 license [14], which supports 
different types of testing in web applications. It is not just a 
single tool but it consists of four tools: Selenium IDE, 
Selenium RC, Selenium Web driver and Selenium Grid. 
Selenium perform tests like functional and regression and 
possess the potential of generating test scripts in programming 
languages like C#, Groovy, Java, Perl, PHP etc. Their tests 
can run on fire-fox web browsers with Windows, Linux, and 
OS X platforms. It can be also integrated with various 
frameworks to provide a hybrid framework, which makes the 
testing simpler [10]. 


3.3.2 Test Complete 

Test Complete is a functional automated testing platform 
developed by Smartbear and operates with test projects and 
project suites. Tests can be recorded, scripted or manually 
created with keyword driven operations and used for 
automated playback and error logging. It is used to create and 
automate different software test types for different 
applications. It has a flexible architecture and used for 
functional testing, database testing, data driven testing. It 
basically comprises of three modules and each module 
contains functionality for creating automated tests on that 
specific platform [17]-[18]. 


Table 2. Selenium Test Suite 


Selenium IDE 


1. Firefox browser 


2.Selenese command 


Selenium RC 


1. RC Server and Client 


plug-in. 2. Java, C#, PHP, Ruby, 
and PERL. 


Selenium Web 
driver 


Selenium Grid 


1.Gridland Grid 2 1. supports many 


featu 
2.Hub-node usage es 


2.Address issues 


3.Automate up to oa Saime 


language 3.Create test scripts for Sbrowsers API 
i 3 User Interface (UT) 
3.Recording and playing 3 Conci 
back on GUI E AS 
programming 
interface 


Table 3. Basic Four Test Complete Module 


Modules of Test Complete 


Desktop applications 


(executed on desktop PC 
running on Windows OS) 


Web applications 


(executed in web browsers embedded in 
desktop) 


Mobile applications 


(executed on Android or iOS 
devices) 
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3.4 Load Testing Tools 


Load testing is the computation of the device with respect to 
its response. It is performed to determine a system's behavior 
under both normal and predicted peak load conditions. The 
respective open and commercial coverage tool are (JMeter 
and HP LoadRunner). 


3.4.1 JMeter 

JMeter is an open source java application testing tool 
developed by “Apache Software Foundation (ASF)”. It is 
used to load test client/server. JMeter is originally used for 
testing Web Application or FTP application. Nowadays, it is 
used for functional test, database server test etc [17], [21]. 
Moreover, JMeter is used in regression testing by generating 


test scripts and provides offline reporting of test results [17]. 


3.4.2. Hp Load Runner 


LoadRunner has various types of Virtual users (Vusers). Each 
type is designed to handle different aspects of client/server 
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architectures [6], [7], [17]. Vuser can be used in any types of 
combination of scenario in order to create a comprehensive 
client/server test. Some of the Vuser types includes: GUI 
(Windows and UNIX), Database (Oracle), and Web (Java). 


4. EVALUATION STUDY 


It is referred to as a benchmarking study. It involves the 
evaluation and comparison study analysis of the selected 
testing tools and techniques from multiple sources. This is 
simply carried out by defining a criterion to provide essential 
application functionality, so as to effectively assess all the 
tools and techniques prior to testing. In this regard, a tool can 
be appropriately selected with their respective techniques 
having satisfied the expected requirement. 


The tables below give the outline of the benchmarked, for the 
tools and techniques using some set of an established criterion 
[10]-[18], [6], [22]. 


Table 4. Software tools comparative analysis 


Criteria Selenium JMeter Testcomplete Hp load runner 
Ease of Easy Easy Experience required. Experienced required 
Installation 
Cost Open source Open source Commercial Commercial 
Purpose a suite of tools to Java application Power Up Desktop, used to test applications, 
Mobile, : : 
automate web browsers designed to load oe measuring system behavior 
test and Web App & performance under load 
across many platforms 
functional 
behavior and 
measure 
performance 
Application Web Web web, desktop and NET Record, Database, 
mobile GUI, Virtual Users, Java, 
support Desktop, Internet 
Application, and Web 
Types of Load Load, functional Unit, GUI and Load and functional 
d ; 
testing done a regression 
regression 
Technical No official technical No official Good technical Good technical support 
Support support technical support support 
Test result HTML XML, HTML and HTML, XML HTTP/HTML 
JSON 
Scripting Java, C#, Python, Java VBScript JavaScript HTTP/HTML, ODBC, 
Language Ruby, JavaScript, PHP. Delphi Script C++ C# SQL.NET, DCOM, 
SAP. 
Environment Windows, MAC, Windows, MAC Windows only Windows and Linux only 
Support (O.S) UNIX, LINUX and Linux 
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Browser Firefox (IDE), chrome, IE (Internet All IE, Mozilla, Opera, 
support IE, Safari. explorer) Netscape, 
Firefox 
Manufacturer Apache 2.0 Apache Software Smart bear Hewlett Packard 
Foundation (ASF) 
Recording Excellent Good Good Excellent 
Ability 
Table 5. Testing Techniques Evaluation 
Testing Level Techniques Tester Scope Time Specification 
Criteria Consumption 
Unit White box Developer Classes High Low level 
design 
Integration White box, Black Developer Multiple classes High, Average, Low & High- 
box and Grey box Low level designs 
Functional Black box Independent tester Entire product Average High level 
design 
Structural White box Developer Entire product High Low level 
design 
System Black box Independent tester Entire product Average Requirement 
Analysis 
Acceptance Black box Client Entire product Average Business 
Requirement 


5. CONCLUSION 


Software testing is the basic activity of software engineering. 
It is an act executed on the software tools with the sole aim of 
detecting and debugging errors and improving software 
quality reliability. To achieve this objective, an appropriate 
techniques and tools are used by a high skilled tester/ 
developer expertise based on their functional requirements. In 
this paper, we have successfully investigated on dynamic kind 
of testing techniques covering the white, black and grey 
boxes, levels of testing and some automated testing tools 
(Selenium, Test Complete, JMeter and LoadRunner). Hence, 
an evaluation and comparative analysis has been outlined 
based on some selected functional criteria to provide a usable 
insight. 
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